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ABSTRACT 
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duration. The group used a grounded theory approach to qualitative 
research, based on the criteria of fit, understanding, generality, 
and control. Rather than present a group conclusion, each member of 
the study group prepared a short » subjective statement summarizing 
the essence of their experience as a participant in the project* 
Sixteen recommendations for program improvements were generated. 
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CHAPTER 1 

ESmRODUCTION 
Purpose of the Study 



The purpose of this study was to give members of the group experience in 
conducting qualitative research. It was felt that an exercise conducting research 
would be more useful to the participants than listening to lectures about 
research. 

Identification of the r^oblem Statement 

This study is to investigate graduate students' experiences in the Center 
for Excellence in Education (CEE) doctoral program in Educational Leadership 
(EDL), at Northern Arizona University. We called our group, out of several in 
the class, the "Flashpots." Specifically, this study will investigate students' 
experiences in the following aspects of the doctoral program: 

L Acceptance into doctoral program 

A. Length of time 

B. Consulting and advisement on the program of studies 

C. Options for program of studies 

D. Access to advisors 

II. Statistic course requirements 

A. Qualitative 

B. Quantitative 



m. Dissertation and comprehensive exams 

A. Requirements 

B. Preparation 

IV. Program Mechanics 

A. On Campus 

B. Off Campus 

V. Other Emergent Factors 

Limitations and Delimitations 

The limitations of a study are restrictions that have been identified from 
the methodology used. Limitations occur when the methodology is not ideal to 
the particular situation. Problems develop when the processes, procedures, or 
populations are not representative of the ideal siti ation creating questions con- 
cerning the external and internal validity of the project. 

The limitations in this study included the absence of a pilot study. Due to 
the time constraints of the course, the Flashpots were unable to test a pilot study 
for this project. This prevented the group from testing the internal and external 
validity of the study. 

The second limitation in the study was the association of the focus group 
participants. The focus group simulation participants were previously intro- 
duced prior to the focus group simulation, and had knowledge of the simulation 
prior to the experience. This can be looked upon as positive or negative. It was 
positive in the respect that the group members were already familiar with one 
another. They were comfortable being asked the questions by the members of 
the group and were able to give open and honest responses. However, it may be 



considered negative because the participants already had preconceived ideas 
about what was to be asked. 

The delimitations of this study were those factors that restricted the scope 
of the study. The delimitations of the study were identified as those factors in 
which participants of the focus group were identified. The participants of the 
focus group were identified as doctoral students admitted into the Educational 
Leadership program in the Center for Excellence in Education at Northern 
Arizona University. Therefore, the results of this study may only be generalized 
to students in the existing program. 

Si gnificance of the Research 

This research project will produce better trained researchers. It will 
provide an experiential learning opportunity for its participants. Additionally, 
the results obtained from the study can be utilized to improve the Educational 
Leadership program in the Center for Excellence in Education. 



CHAPTER 2 
REVIEW OF THE LITERATURE 

A review of relevant literature was conducted prior to and during the 

course of this study. The purpose of this review was to utilize background 
reading to provide the project with the resources enumerated by Strauss and 
Corbin (1990): 

1. To stimulate theoretical sensitivity. 

2. To provide secondary sources of data. 

3. To stimulate questions. 

4. To direct theoretical sampling. 

5. To provide supplementary validation. 

In addition to a short survey of the literature in qualitative research, particularly 
in the area of focus group driven research, the study was derived as a partial 
replication of a previous project conducted by Packard and Dereshiwsky (1990) 
at the same site. 

Validity in qualitative research, the degree to which qualitative proce- 
dures really measure what they propose to measure, has been the topic of a 
certain degree of debate in the literature. The focus group method utilized in this 
study can be the subject of most of the criticisms and kudos made with regard to 
validity considerations in qualitative methods in general. Beginning with 



Cambell and Stanley's work in 1963 (Schofield, 1989), the debate over external 
validity in qualitative research has focused on the concep'. of generalizability: 
the discontent with perceived weaknesses in many quantitative techniques. 
Cook and Campbell (1979) subdivide the validity issue into internal and external 
validities, each dependent upon the approach to cause and effect taken by the 
study and its critics. Further, they recognize considerations of construct validity 
and statistical conclusion validity, all of which are tied to a priori assumptions 
regarding cause and effect. Since the issues of which validities are violated, 
exactly when and why this occurs, and why they may vary so much from study 
to study, Krueger recommends that researchers, critics and consumers rely 
primarily on the face, predictive and convergent validity of the stud. In other 
words, does the study appear sensible? If so, are the study's conclusions useful 
in predicting behaviors and in agreement v^th other useful works in the area? 

Because of the difficulty in establishing exact equivalences across 
qualitative studies, some theoreticians (Cuba & Lincoln, 1981: cited in Schofield) 
have proposed the adoption of a new term, fittingness, to describe approximate 
equivalence between research situations. In the study of educational situations, 
Schofield (1989) proposes that generalizability may apply to 1) the study of what 
is, 2) the study of what may be, and 3) the study of what could be. Regarding 
approaches to the study of what is, the chief concern of the current paper, 
Schofield recommends 1) studying the typical, and 2) performing multisite 
studies, Krueger (1988) observes that the opportunities for error attendant upon 
over generalization are readily apparent in focus group driven research. In 
important or crucial situations, he recommends that multimethod approaches, 
(including focus groups), will increase the chances of accurate prediction. 

The current project relies upon a group developed protocol. Guidance in 
this area is supplied by "Case Study Research: Design and Methods," (Yin, 1989). 
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Prior to protocol development, Yin recommends that research team members 
clarify why the study is being done, what evidence is being sought, and what 
variations or contradictions in the evidence might be expected. Protocols based 
upon these considerations would then be developed as a team effort. Glesne and 
Peshkin (1992) advise researchers to develop questions which fit the topic, 
illuminate the phenomenon under investigation, and be anchored in the cultural 
reality of the informants' lives. In addition, during the pilot (and subsequent) 
stages of the enquiry, Glesne and Peshkin recommend that careful consideration 
be given to critiques of the protocol by colleagues, informants, and advisors. 

Morgan (1988) observes that the use of focus groups has historically been 
common in marketing, but is relatively new to social science research. He 
observes that the focus group method is desirable for obtaining concentrated data 
from a series of interactions by group participants. This type of collection, 
particularly useful in obtaining information regarding attitudes and cognitions, 
provides the kind of "thick" descriptions recommended by Schofield (1988) as 
the major benefit of qualitative approaches. 

A phenomenon attendant upon focus group driven research is that group 
interactions replace researcher control (Morgan, 1988). The result, however, is a 
two sided coin; the information derived from focus groups is the result of the 
interactions of the participants and is: 1) not necessarily focused on the 
researcher's concerns and 2) not necessarily the same information which the 
researcher might have obtained by interaction with each focus group member 
individually. 

"Coding is a progressive process of sorting and defining and defining and 
sorting those scraps of collected data ... that are applicable to our research 
purpose" (Glesne & Peshkin, p. 133). The coding of focus group generated data 
is a process of analysis and synthesis consisting of "examining, categorizing. 
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tabulating, or otherwise recombining the evidence to address the initial 
propositions of a stud/' (Yin, 1984, p. 99). Given any set of data, there are likely 
to be numerous ways to code it The choice of problem statement, identification 
of subproblems and selection of study participants, as well as known and 
unknown biases and presuppositions on the researcher's part, are likely to have 
direct and indirect impacts on the choice of a coding system. In order to 
maintain a claim to reasonable objectivity and systemic integrity, Krueger (1988) 
recommends that certain rules be followed while analyzing focus group data: 1) 
pay attention to individual words and tag phrases, 2) take context into account, 
3) note internal consistency and inconsistency, particularly shifts in position, 4) 
distinguish specific, experience-based responses from vague and impersonal 
responses, 5) isolate the major ideas in the discourse, (avoid not seeing the forest 
because of the trees), and 6) maintain a sense of the purpose, type, and scope of 
the final report. 

Miles and Huberman (1984) discuss twelve methods for deriving 
meaningful conclusions from qualitative data. They include: counting, seeing 
plausibility, clustering, making metaphors, differentiation, factoring, detecting 
intervening variables, and making theoretical and conceptual coherence. 
Although the process of drawing conclusions traditionally occurs near the end of 
a study, in reality a great deal of the researcher's anticipated decisions can be 
inferred from the earliest stages of conceptualizing and planning the project. The 
holistic fallacy, elite bias and the process of goin^ native (Miles & Huberman, 
1984) may intervene to skew results at any stage of the process but are 
particularly prone to affect the coding, analysis and conclusion making process. 
Careful selection of sites and participants as well as thorough review of the 
literature and care in the planning stages of the project (Yin, 1984; Krueger, 198fi; 
Schofield, 1989) should be helpful in reducing the chance of false conclusions in 
the final report. 

JO 



CHAPTERS 
METHODOLOGY 

The Problem Statement 

This study is to investigate graduate students' experiences in the CEE 
doctoral program in Educational Leadership. Specifically, this study will 
investigate students' experiences in the following areas of the doctoral program: 

L Acceptance into doctoral program 

A. Length of time 

B. Consulting and advisement on the program of studies 

C. Options for program of studies 

D. Access to advisors 

IL Statistic course requirements 

A. Qualitative 

B. Quantitativo 

in. Dissertation and comprehensive exams 

A. Require-nents 

B. Preparation 

IV. Prt>);ram Mechanics 

A. On Campus 

B. Off Campus 

V. Other Emergent Factors 

9 

11 



Population and Sample 
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The participants for this focus group consisted of seven part-time students 
pursuing doctoral degrees in Educational Leadership at the Center for Excellence 
in Education at Northern Arizona University. All participants, at the time of the 
study, were members of EDR 735 Qualitative Research, Summer Session H, 1993. 
See the table on page 12 for a demographic description. 

The participants were officially accepted into the program beginning from 
Spring 1989 through Spring 1993. The length of time from earning a prior degree 
ranged from one to twenty-five years. Their educational background included: 
education, speech pathology, higher education, social work, and medicine. Four 
m'-mbers were part-time students who completed a significant part of their 
program at locations such as Phoenix, Tuscon, and Pt, Loma. The remaining 
three members were completing their studies on campus in Flagstaff. 

All participants were employed at the time of the study. Five were 
employed in classified position, one in an unclassified position, and one in an 
administrative position. The range of time in their position at the time of the 
study was from two to thirteen years. 

Methods of Qualitative Data Collection 

The method of data coding and collection was derived from procedures 
proposed by Miles and Huberman (1984) The members took turns as 
questioners and interviewees. A tape recorder was utilized to ensure accurate 
records of the proceedings. One member pro vided transcription services and 
other members assisted in script taking. 

Two focus group sessions of approximately one hour duration occurred 
on July 21 and 22, 1993. After these sessions, the audio recordings were 
transcribed, the data coded, and the results tabulated in the matrices found later 
in this chapter. 
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Focus Group Questions and Probes 

The following is the list of questions that were utilized in the focus groups 
in order to obtain the data for our study. 

1. Tell us about the mechanics of your acceptance into the program 

in terms of the process 

in terms of advisement on your program of study 
in terms of options and alternatives 
in terms of access to advisors* 

2. The statistic courses appear to raise concerns for many students. What is 
your response to the requirements? 

qualitative versus quantitative 
in terms of usefulness to you. 

3. Comprehensive examinations raise concerns for many students. What is your 
reaction to them? 

in knowing what they are 

in terms of preparing yourself to take them 

in terms of your anxiety level about taking them» 

4. Doing a dissertation raises concerns foi many students. What is your reaction 
to the dissertation? 

in terms of knowing what to do 

in terms of getting assistance from your advisor 

in terms of it meeting the program's needs or your needs. 

5. How have you experienced this program as an on campus or off campus 
student? How have you experienced this program as a part-time or full-time 
student? 

in terms of course offerings 

in terms of access to your advisor 

in terms of access to research literature 

in terms of meeting the residency requirements. 
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CHAPTER 4 
DATA ANALYSIS 

Grounded Theory 

The Hashpots used a grounded theory approach to qualitative research in 
their study of CEE doctoral students in Educational Leadership. This method 
''uses a systematic set of procedures to develop an inductively derived grounded 
theory about a phenomenon" (Strauss and Corbin, 1990). The data collected 
from the focus group simulation consisted, theoretically, of the reality of the 
subjects in the study. The relationships between the focus group simulation 
participants and the study (i.e. experiences in the doctoral program) were gener- 
ated and tested within the focus group simulation, allowing theory to be discov- 
ered. 

The focus groups' grounded theory approach was based on the following 
criteria as stated by Strauss and Corbin: 1) fit, 2) understanding, 3) generality, 
and 4) control (Strauss and Corbin, 1990). The purpose of the qualitative study 
completed by the Flashpots was to investigate students in the CEE doctoral 
program in Educational Leadership. Their experiences in the program in specific 
areas listed, determined the strengths, weaknesses, and recommendations for 
improvement for future doctoral students in the program. 
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A focus group consisting of existing doctoral students investigated these 
areas of concern. Demographics were obtained from the focus group members. 
Data collected from the demographic survey displayed a well-balance of stu- 
dents in terms of age, ethnicity, location, and stage in their program. By inter- 
viewing these students already m the doctoral program, the Flashpots were able 
to substantiate our data and make a "fit" between the everyday reality of the 
substantive area and the carefully induced diverse data (Strauss and Corbin, 
1990). 

The focus group in this study represented real students in the Educational 
Leadership doctoral program. Therefore, this well-constructed grounded theory 
represented reality and will make sense to not only the subjects in the study, but 
other students, faculty, staff, and administrators in the program. 

The data collected was broad-based and conceptual allowing the theory to 
be generalized to other students within the program. The focus group questions 
and probes consisted of those which may be used for other students in the CEE 
doctoral program in Educational Leadership, permitting the generalization of 
theory. 

Finally, the hypotheses stated in the study proposed relationships among 
the concepts which are systematically derived from the actual data related to the 
events (Strauss and Corbin, 1990). The concepts of the doctoral program were 
stated in terms of questions and probes addressed in the focus group. As previ- 
ously stated, concepts including demographic information about the subjects 
assured the balance of agp, ethnicity, location, and level of progress in the doc- 
toral program. This allowed the theory to provide control with regard to action 
toward the events of the study. 
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Ouesrion#l: Narrative 
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Acceptance Into Program: Perceived Strengths 

Five of the seven focus group members stated that the process of being 
accepted into the doctoral program went rather smoothly once they received the 
information. As one participant stated, "The interview committee came to Point 
Loma to hold interviews. The people at Point Loma were also very helpful with 
putting thj? portfolio together." 

A number of respondents commented on the strengths of the advisement 
process in the program. One participant noted that she was advised by a profes- 
sor that the doctoral program was the "next step to go" after her master's pro- 
gram. Another had been advised to apply for the program at NAU because it 
was viewed as one of the top programs in the country, One participant men- 
tioned that the master's program in EDL allowed ease into the doctoral program 
in EDL. In one instance, a participant had taken EDL 600 Leadership Skills in the 
master's program. A portfolio assignment was made in that particular course 
and she was advised to continue building on the portfolio, which would assist in 
the preparation for the doctoral portfolio. Several stated that they were sent a 
packet upon request of admissions information which included 2 to 3 pages of 
procedures for application, and what should be included in the portfolio. This 
allowed the application process to go rather smoothly. 

Acceptance Into Program: Perceived Weaknesses 

Regarding the process of being accepted into the program, a few members 
stated that they did not receive any information or instructions on the applica- 
tion process or the portfolio, unlike other members of the group. This demon- 
strated some inconsistencies in the application process. One particular member 
discussed the fact that he lost 9 months in the application process because he was 
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not given information about his advisor who was no longer at the university. He 
found out that his advisor was involved in a textbook scam by reading it in the 
newspaper. No one told the student this when he called. "I had a week and a 
half to put together a second portfolio because there were some problems with 
the person that was appointed as my advisor." 

It was discussed that there is some lack of clarity and possible redundancy 
in terms of the CEE vs. Graduate College application requirements and pro- 
cesses. The packet of information some students received on the application 
process led them to believe that official transcripts need to be induced in the 
portfolio as well as sent to the graduate college with their application. 

One participant felt that there were some inconsistencies in the portfolio 
evaluation point system. As stated by one member, . . one professor gave me 
zero points for creativity. In my life I have been a graphic artist and have been a 
musician all my life, and I felt that was totally inappropriate as far as giving me 
no points in creativity." 

In regards to the actual interview procedure, several participants stated 
that they drove several hours to attend a 20 minute interview. One person stated 
that there were only 2 professors in the interview. 

In terms of advisement into the program, a few members perceived the 
process as needing improvement. Comments were made that they received no 
advisement on the application process or portfolio, and in one case, an advisor in 
the master's program gave one particular individual incorrect information about 
the process. Other members had to search for information about the program, 
which was a problem. 

Unlike one member in the group, the others stated that they did not have 
any courses in the master's program or anything that assisted in the preparation 
of the portfolio. They were given a list of items to include, but information was 
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not available on the contents or format of the portfolio. 

When discussing access to advisors in the program, participants felt that 
access was difficult mainly because there were too many (200) students in the 
doctoral program. It is too difficult to accommodate that many students in the 
program. It was a concern that advisor are not assigned to students until after 
they are accepted into the program, and in some instances off-campus, advisors 
are not available at all. Another primary concern was the fact that no one in the 
Center has an up-to-date data-base on all the students that show where they are 
in their program. A lot of information is missing, not available, or is not current. 

Acceptance Into Program: Rficommendations for Improvement 

The group participants made clear recommendations for improvement of 
the CEE/NAU doctoral program in Educational Leadership. First, consistency 
in the application process needs to be addressed. All students should receive a 
uniform packet of what the requirements are to get into the program. The packet 
should include information regarding the portfolio items. Participants felt that 
follow-up information about the application pre ess (i.e. step-by-step feedback) 
should be provided. It was also suggested that clarification needs to be made 
concerning the CEE and Graduate College application requirements and pro- 
cesses to reduce the perceived redundancy. 

The participants who are located off-campus felt that interviews for some 
of the off-campus students should be held in a more centrally located area such 
as Phoenix. This would eliminate the long and tiring drives for 20 minute inter- 
views. 

Advisement into the program was also a key focal point. The participants 
felt that adequate advisement on the application process and portfolio should be 
available and accessible to everyone applying to the program. Advisors should 
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be available off-campus as well, to assist the off-campus students that do not 
have the benefits of being on-campus to access needed information. It was also 
suggested that courses in the master's program should assist in the process as 
they did for one participant. 
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EDL DOCTORAL PROGRAM EXPERIENCES MATRIX 
Question #la 



TOPIC 


EXPERIENCES 


SPEQFICS 


PERCEIVED 
STRENGTHS 


PERCEIVED 
WEAKNESSES 


RECOMMENDA- 
TIONS 


Acceptance 
into program 


Process 


Time 
Portfolio 
Application 
Interview 


Portfolio and 
application 
process went 
smoothiy for 
some members 

Interview 
committee came to 
Point Loma 


Several hour drive 
for short interview 

Only 2 professors at 
interview 

Inconsistencies in 
point system of 
portfolio evaluation 

Lack of instructions 
for portfolio 

L:>st 9 months due 
to advisor leaving - 
department failed to 
notify 

Redundancy and 

lack of clarity 
regarding CEE and 

Graduate college 
requirements 


Hold interviews at 
off-campus 
locations 

Provide a uniform 
packet of program 
requirements to all 
students 

Make application 
process consistent 

Provide step by 
step feedback at all 

points of 
application process 

Clarify 
requirements and 

applications 
processes of CEE 
and Graduate 
College 
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EDL DOCTORAL PROGRAM EXPERIENCES MATRIX 
Question #lb 



TOPIC 




SPEQFICS 


PERCEIVED 


PERCEIVED 


RECOMMENDA- 


EXPERIENCES 


STRENGTHS 


WEAKNESSES 


TIONS 


Acceptance into 


Consulting and 


Application 


Master's in EDL 


No advisement 


Provide adequate 


program (cont.) 


advisement 


information 


allowed easy 


on application or 


advisement and 








transition into 


portfolio process 


assistance on the 






Portfolio 


Ed.D. 




application and 










Lack of 


portfolio process 






Program of 


Informative 2-3 


preparation for 








Studies 


pages of 


portfolio 


Standardize 








instructions for 




packet 








portfolio 


Information 


information sent 








provided 


difficult to ob ain 


to students and 








regarding program 


advisors 








EDL 600 portfolio 












assignment 


Advisor gave 










prepared for 


incorrect 










Ed.D. application 


information 












regarding 










CEE/NAU 


program 










EDLEdD. 












perceived as one 












of top ten in 












country 
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EDL DOCTORAL PROGRAM EXPERIENCES MATRIX 
Question #lc 



TOPIC 


EXPERIENCES 


SPECIFICS 


r EiKCcl V E,U 


WFAKNFSSFS 


TIONS 


Acceptance into 


Access to advisors 


On campus 




High number of 


Provide 


program 






Point Loma 


students decreases 


pre-advisement 




Off campus 


advisors very 


accessibility to 


on portfolio 








helpful 


advise 


and admission 






Prior to 






processes 






admission 




jrVtlVlSOra 












frequently not 


Provide master 










available off 


courses or 










campus 


seminars to 












assist in 










No advisement 


admissions 










available until after 


process 










acceptance 














Increase 










Lack of accurate 


accessibility and 










and upto-date 


standardize 










database detailing 


information about 










student's standing 


programs and 










on adn\ission 


processes 










process 
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Question #2: Narrative 



Qualitative Statistics: Specifics 

Two of the positive responses were aimed at exactly that; the qualitative 
statistics courses were positive experiences. Another positive response indicated 
that the respondent liked their professors. The final comment found under 
qualitative /specifics was that the respondent now had the tools to do a 
qualitative investigation in an area that had previously been done quantitatively. 

Qualitative Statistics: Perceived Strengths 

Qualitative statistics is much more relevant to the EDL dissertation 
according to two focus group members. Two others indicated that the 
applicability to their area of interest is just incredible. Another remarked that 
qualitative is very nicely related to quantitative statistics. One group member 
indicated that the ability to interrelate the two areas was an asset 
(multimethodolgy). 

Qualitative Statistics: Perceived Weaknesses 

There were no comments on any perceived weaknesses regarding 
qualitative statistics in the EDL doctoral program. 

Qualitative Statistics: Recommendations For Improvement 

The entire focus group agreed that there should be more emphasis on 
qualitative statistics in the EDL doctoral program because it is so much more 
relevant to the type of research that is done in Educational Leadership. The one 
suggestion was that qualitative statistics should be emphasized in the area of 
educational research. Its relevance to the type of research that is done in 
education is seemingly inveighed. 

28 



23 

Quantitative Statistics: Specifics 

There were two people in the focus group that had positive experiences 
with statistics courses. Another respondent liked their quantitative professors. 
One of the naembers remarked about having statistics anxiety and there didn't 
seem to be much they could do about it other than bite the bullet and muddle 
through the course to obtain a grade of C. 

One individual commented that taking the quantitative courses on the 
computer with the professor who wrote the book was a very positive experience. 
As a carry over, he/she is extremely happy with the professors he/she now has 
for qualitative statistics. Finally, two group members agreed that taking 
quantitative statistics without computers should be required. They both felt that 
everyone ought to "sweat bullets" just to get the feel of quantitative statistics. 

Quantitative Statistics: Perceived Strengths 

There was general agreement that the introductory quantitative statistics 
course was a good foundation for future course work. Two members stated that 
they thought it was relevant to the doctoral program course of studies. 

Quantitative Statistics: Perceived Weaknesses 

One respondent related a story of how the quantitative instructor seemed 
to go out of his way to make things as difficult as possible. The story goes on 
that the professor would come in to class and begin to write "stuff" on one end of 
the chalk board and go around the room until all of the chalk borrds were full of 
problems and equations. The professor would explain very little and then erase 
it all and start again. 

Another group member related how his statistics anxiety came to bear 
when taking the quantitative introduction course in a summer session course. 
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Tkiee or more chapters were covered in one day whidi overwhelmed the 
student. Furthermore, at the lime of the second of three exams the student had 
the flu, did poorly on the exam, and subsequently lost his 4.0 GPA because of the 
C he received for the class. 

One respondent expressed concern that quantitative statistics was difficult 
to apply to real-life situations. On the other hand, another person remarked as to 
the deftness with which he could apply quantitative measures to his current 
project. 

Quantitative Statistics: Recommendations For Improvement 

There was general consensus that quantitative statistics courses should 

not be taken during summer session classes, especially if one has statistics 

anxiety. The interference from anxiety and/or illness could have a deleterious 

effect on the student's class performance. 

Two focus group members indicated that quantitative statistics should not 

be taken with the aid of computers. Finally, one group member felt that taking 

quantitative statistics using a computer program was perfectly acceptable, and 

should be left to one's own personal choice. 
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EDL DOCTORAL PROGRAM EXPERIENCES MATRIX 
Question #2 



TOPIC 


EXPERIENCES 


SPECIFICS 


PERCEIVED 
STRENGTHS 


PERCEIVED 
WEAKNESSES 


RECOMMENDA- 
TIONS 


Statistics Course 
Requirements 


Qualitative 


2 students had 
positive 
experiences 

Research was 
adapted from 

previous 
quantitative 

methods 


Good professors 

Highly relevant 
for dissertation 

Relates well to 
quantitative 
research 

Integrated well in 
multimethods 


Vague 

Not helpful for 
comprehensive 
exams 


Increase number 
of course 
offerings in 
Qualitative design 
i.e. 

Focus Group, 
Interviewing, 
Case studies and 
Ethnography 




Quantitative 


1 student had 
negative experience 

with statistics 
in summer session 

2 students had 

positive 
experiences 


Good professors 

Useful for 
evaluating and 

critiquing 
research articles 

Relevant 


Too intense for 
Slimmer session for 
many students 

High-level of 
anxiety induced 
with no remediatior 
available 

Difficult to apply to 
dissertation 


Inaease use of 
computers in 
statistics 
courses 
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Question #3: Narrative 
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Comprehensive Examination: Perceived Strengths 

The most positive factor regarding the exams that most members men- 
tioned was the formation of study groups. These study groups help ease the 
anxiety created by the exams. Several members mentioned that many of the 
professors are willing to provide study guides such as the ones on comparative 
education written by Dr. and the videotape by Dr. D. 

The take home portions of the exams (i.e. article critique and review of 
literature), were thought to be positive aspect of the exams. Many members 
thought these portions of the exams better tested the student's true abilities. 
Additionally, some members thought the experience of writing the 20-25 page 
paper would help considerably in the completion of their dissertation. 

Comprehensive Examination: Perceived Weaknesses 

All members had negative comments on the current way comprehensive 
examinations are given. Most felt that the time constraints were not related to 
tasks experienced in the real world. Many expressed the need to cram for the 
exams rather than develop a comprehensive understanding of the 
materials.. ."you just sit there and you regurgitate/' 

All members felt that there was too much of a mystique surrounding the 
exams. One member expressed her frustration with the exams as the "Big Se- 
cret/' Another complained about the lack of information provided regarding 
the exams. 



Comprehensive Examinations: Recommendations for Improvements 

It was suggested that a handbook be provided with a step-by-step guide 
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of what to study and how to prepare for the comprehensive exams. A checklist 
should be included so that students know where they stand regarding the exam. 
Many members thought the timed portions of the comprehensive examination 
should be eliminated and be replaced with take home exercises. This would 
allow the students to demonstrate their true comprehension of the materials. 
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EDL DOCTORAL PROGRAM EXPERIENCES MATRIX 
Question #3 



TOPIC 


EXPERIENCES 


SPEOFICS 


JrliiKdiii V ILi/ 


W b A KIM li^aali^o 




Comprehensive 


Requirements 


Most students 


Informal study 


Secretiveness 


Improve 


Examinations 


Preparation 


expressed high 


groups 


Unnecessary anxKty 


communication 






anxiety 




created 


with students 








Helpfulness of 




regarding comps 






Most students felt 


certain professors 


Lack of knowing 


i.e., a step-by-step 






too little 


to provide 


what classes to take 


handbook 






informations was 


information 


to prepare for comps 








made available 


regarding the 


and when to 


Create a database 








comps 


schedule comps 


and update it each 






Many students felt 






semester to indicate 






they were 


The research 


Lack of written 


student's progress 






inadequately 


portion of comps as 


materials on comps 


toward comps, 






prepared 


a "take home" 




residency, and other 








exercise 


Lack of validity 


graduation 






Some students 




tests recall 


requirements 






questioned the need 




instead of 








ind/or validity of the 




comprehension 


Send student a 






comps 






printout each 










Inappropriate time 


semester 










limit for writing 














Change all of 










Doesn't relate to 


comps to *'take 










writing in real world 


home" exercise 



Question 4: Narrative 
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Dissertation: Perceived Strengths 

The focus group's participants* responses to the questions on the 
dissertation were mostly positive. Negative responses and recommendations for 
improvement of the program were also made. 

The question regarding advisement for the dissertation received some 
very positive results. One of the participants remarks, " I feel that I have received 
good direction in terms of what I need to do for my dissertation. I feel that I 
have had a lot of direction by inquiring." Another participant's response as to 
the selection of a topic for the dissertation stated that the advisor did not 
provide much assistance with the selection of the topic. The advice given was, 
"Do it on something you really care about, that you are willing to work on for a 
long period of time." Additional advice given was, "It will come to you." 

Dissertation: Perceived Weaknesses 

There was consensus among the group that there is much confusion 
during the early stages of the students' experience in the program. One response 
given by an advisor to a participant's question was: " Many of the questions you 
have about the program will be answered as you progress through the 
program." 

Dissertation: Recommendations for Improvement 

It was suggested that additional assistance and courses be provided to 
those students off-cai apus. It was also recommended that the dissertation 
seminar course be offered earlier in the students' programs in order to provide 
them with information needed for the rest of their course of studies. 
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EDL DOCTORAL PROGRAM EXPERIENCES MATRIX 
Question #4 



TOPIC 


EXPERIENCES 


SPECIFICS 


Jr JEiIVV^JCi V nu/ 


WFAKNESSES 


RECOMMENDA-I 
TIONS 


Dissertation 


Process 


Knowing the 


Advisers and 


Insufficient 


Provide 






procedure 


faculty willing to 


advisement with 


additional off 








assist 


dissertation topic 


campus assistance 










and process 










High accessibility 




Modernize format 








of faculty 


Uncreative and 


requirements to 










nrimitive 


acknowledge and 








High accessibility 


format 


utilize computer 








to literature 




typesetting 










Confusion 


capabilities 








Library staff 


Students need to 










assistance 


constantly inquire 


Add pre- 










about details 


dissertation 












seminar course at 










Time frame 


beginning of 










inflexible 


program of 












studies to give 










21/2 week courses 


students a 










cause difficulties 


head-start 












Teach additional 












courses off 












campus 
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Questions: Narrative 

Location: Perceived Streng ths 

Two of the six participants were on-campus students. They felt that they 

had good access to information, and the faculty were willing to assist with their 
needs. 

Four of the six participants were considered off-campus students. Three 
of the foiu" responded positively to the courses that were available off-campus. 
One of the Phoenix students stated, " I haven't had any problems at all with the 
course offerings in the valley. I was accepted into the program, I believe in 1991 
and again in 1993. I should be finishing my course work this fall, and this is as a 
part-t^Tne student." 

Location: Perceived Weaknesses 

One of the on-campus students felt that there was a problem with the 
sequence of courses offered. The off-campus students expressed the same 
concerns as well as concerns that their program of studies had not been signed 
after being in the program for four years. It was also unclear how to meet the 
requirements as an off-campus student. 

Location: Recommendations for Improvement 

It was suggested that the sequenced of courses offered on-campus be 
clarified to a certain degree. Off-campus students recommended thdt additional 
faculty be provided to teach off-campus courses, and that clarification needs to 
be made concerning residency requirements. 
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EDL DOCTORAL PROGRAM EXPERIENCES MATRIX 



Question #5 



1 Virlv- 


EXPERIENCES 


SPECIFICS 


PERCEIVED 
STRENGTHS 


PERCEIVED 
WEAKNESSES 


RECOMMENDA- 
TIONS 


On campus 


Course 
offering 

Access to Advisors 

Literature Access 

Residency 
Requirement 


Sequence 

Advisement 
Quality& Quantity 
Many questions 


Flexibility 

Faculty willing to 
assist 

Good access 


Sequence 
difficulties 


Clarify sequence 


Off campus 


Course 
offering 


Sequence 


Eexibility 


Sequence 
difficulties 


Add faculty for off 
campus program 




Access to Advisors 

Literature Access 

Residency 
Requirement 


Advisement 

Quality& 
Quantity 

Many Questions 


Faculty willing to 
assist 

Good access 


Program of studies 
unsigned after 4 
years 

Requirements 
unclear how to 
meet 


Clarify 
how to meet 

residency 
requirements 
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CHAPTER 5 

SUMMARY, CONCLUSIONS, RECOMMENDATIONS AND IMPUCATIONS 

Summary 

Rather than include a conventional summary of results and recommenda- 
tions for future research, the study participants have decided to attempt a sum- 
mary of this project from a phenomenological perspective. It seemed to the 
participants that the variety of backgrounds, ages, ethnic backgrounds and life 
experiences of the study group members might provide a stimulating body of 
insights into what was, empirically at least, a shared experience of qualitative 
research. Rather than present a group conclusion synthesized by an editor, each 
member of the study group was asked to prepare a short, subjective statement 
summarizing, if possible, the essence of their experience or experiences as a 
participant in the project. 

The group anticipates a high degree of agreement and disagreement 
among participants. By sorting these points of agreement and disagreement (in 
light, of course, of their own internal coding systems) readers may infer for 
themselves whether or not the Flashpots had captured the essence of a qualita- 
tive research experience (represented by points of agreement) or if their essential 
personhoods, plus their multicultural, multiethnic backgrounds, caused the 
creation of experiential and remembered parallel realities (points of disagree- 
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ment). Shared essence is the foundation of reality in the phenomenological 
approach. To paraphrase Edmund Husserl (1913-1962), the founder of the phe- 
nomenological school, 'We can only know what we experience by attending to 
perceptions and meanings that awaken our consdous awareness" (Patton, 1990). 

Participant Y 

During the earliest stages of the group's existence, I formed a strong 
impression that our project was doomed to be a typical administrators-in-sum- 
mer school group project. This is to say, everything would be put off until the 
day before the due date and then hastily slopped together at the last minute by 
the two or three most anxious members of the group. Now, as we approach the 
end of our project, I am already looking back on it with a great deal of nostalgia, 
primarily because everyone has worked hard on individual and shared labor, no 
one has taken a free ride, and we have a reasonable prospect of coming in under 
the deadline. 

Reviewing the tables, I am impressed, not by the amount of "bellyaching" 
(which is what I anticipated would be the essence of our focus group), but by 
how many positive points were discovered about the doctoral program and how 
many good suggestions for improvement the group was able to generate. 

We have worked so well together, considering that we were strangers just 
three weeks ago, that I am really quite sorry to see our time together end so soon. 
Maybe somewhere in this sentiment is what is truly the good aspect of the doc- 
toral program. Maybe this is also the good aspect of qualitative research. I have 
spent so many lonely hours huddled in front of computer screens, crunching 
numbers, that this way of exploring reality has been like the opening of a huge 
door to an exciting new place. 
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Participant W 

The entire process of the focus group simulation was a very positive 
experience. The group as a whole worked very well together and each partici- 
pant played a significant role in the success of the group. The group leader 
acted as a moderator, allowing leadership of each participant to come through. 
Specific tasks were assigned (voluntarily) so that each participant was able to 
focus on a particular area of the assignment and carry nut that task. 

The focus group was very interesting. I felt very comfortable during the 
entire process and didn't feel intimidated by anyone in the group. It was a very 
relaxed atmosphere and I felt that the questions were answered openly and 
honestly. We went in knowing that names would not be used in the report, and 
the participants appeared to feel comfortable with that. 

I gained knowledge about qualitative research by participating in the 
focus group. Participating in such a group enabled me to put readings on quali- 
tative research into perspective. This experience allowed me to investigate areas 
of the CEE doctoral program in Educational Leadership that may need improve- 
ment for future doctoral students. This experience will assist others and myself, 
in gaining knowledge about the program. It will also encourage us to be sensi- 
tive to the needs of future students applying for the program. With an under- 
standing of the perceived strengths, weaknesses and recommendation, we will 
be able to take action to make the processes of the program easier for students. 

This project also allowed me to gain knowledge and experience in qualita- 
tive research that will assist me in future research projects, including my disser- 
tation. I feel comfortable with the idea of setting up focus groups for future 
research projects. As a focus group participant, I was able to see, as an insider, 
the process of the focus group, allowing me to be sensitive to those I may recruit 
into a focus group in the future. 
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FartidPant X 
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As partidpant '^X", this research project was a valuable learning experi- 
ence. This was my first opportunity to become involved in a focus group process. 
Research Design, is where I first heard about focus groups. 

With an academic and experiential background in the social sdences, 
qualitative research utilizing focus groups made a lot of sense to me from the 
beginning. It seemed to get at the quality of the experiences that might be the 
subject of the research. The colleagues with whom I worked, helped make this 
experience a good one because each expressed an interest, and the tasks were 
easily divided up utilizing the various skills and interests of the group members. 
Due to my experience in interviewing, I volunteered to draft the questions for the 
Demographic Profile and the Focus Group Questions and Probes. Also, because 
of my secretarial skills, I volunteered to transcribe the audiotapes. 

From this research project, I have learned that focus groups can be hard 
work in terms of developing the research questions, the focus group questions 
and probes that will illidt quality information, transcribing the audiotape, accu- 
rately coding the material, and generating the final report. However, I have also 
learned that the depth of information supports its own validity. Certainly in this 
case, there is no reason for participants to provide other than genuine informa- 
tion because it relates to personal experiences, information which only they 
could provide. 

Participant R 

The experience of partidpating in a group project to investigate the stu- 
dents' perceptions and experiences in the CEE doctoral program at NAU has 
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given me an invaluable experience in developing a problem statement and con- 
ducting a focus interview. Participating in the focus group as both interviewer 
and as interviewee has somewhat eased my anxiety of conducting the same for 
my dissertation. Many anxieties of the focus interview process, the coding pro- 
cess, and the development of matrices, have been eased considerably. I intend 
to use a focus group for my dissertation and the process seems to be much more 
dear than prior to participating in this project. Working with a group of profes- 
sionals that composed our team was also invaluable. Each of the members par- 
ticipated and contributed equally. No one member of the team was asked to 
contribute more than their share. 

The experience of actually developing the 'Problem Statement," coding, 
and developing matrices for the two of the five questions used in the focus 
group, provided me with an experience that would not otherwise have been 
available to me prior to doing the ''real thing" for my dissertation, 

A group of professionals, such as our team, enabled us to conduct the 
focus interview in a professional manner. I hope that other doctoral students are 
fortunate enough to experience the process as I experienced it, and gain the 
invaluable knowledge that I did. 

Participant Z 

This project allowed me to participate in the creation of a focus group, set 
and define the delimitations of the study, adopt the role of interviewee and 
interviewer, code and analyze the data collected, and share the results with 
others. 

The experience of being a member in the focus group was positive. Many 
of the issues that were discussed had personal meaning to me. Overall, the 
group process went smoothly. There were no apparent conflicts and all 



members provided a good measure of input. 

As a student who is preparing to write a dissertation by utilizing focus 
group methodology, this project was highly useful. In a certain manner, the 
projects combmed Dewey's philosophy of learning by doing and Eisner's advise 
of becoming artistic critics of evaluation. 

It vrauld be interesting to do a follow-up study of recently enrolled CEE 
doctoral student to see if their perception of entry into the program was similar 
to those of our focus group. This follow-up study could help to verify the 
improvements that needed to be made in the admissions process. Hopefully, 
the data collected will be shared with the administrators of the NAU/CEE 
doctoral program since it represents the experiences of a wide cross-section of 
the population. It would heighten the value of this project to know that the data 
and resiolts would be used to strengthen the CEE doctoral program rather than 
being just another project that we completed as part of our journey toward the 
Ed.D. degree. 

Participant U 

By participating in the study, I felt I had something of value to contribute 
to future doctoral students in Northern Arizona University's Educational 
Leadership program in CEE. Although I have not experienced any difficulty in 
the process of being accepted into the program, I have experienced being "lost" in 
the shuffle of students. As I progress in the program, however, I am beginning 
to feel more comfortable in knowing exactly what is the next step is in the 
dissertation path to the "Doctor" stage. 

During the simulation focus group process, I felt the participants were 
genuine and had a sincere attitude toward the process. Since the participants 
were fairly new to the to the program, there was a camaraderie approach to 



discussing and answering the questions that were asked. 

From my perspective, the comprehensive exam was a major concern of all 
the participants since none of the participants had yet taken the exam(s). My 
concern was, why make it a mystery learning experience as opposed to a 
"mastery" learning experience, as Dr. G. would say. I felt that if all the students 
were informed of the contents of the exam, the students would be better 
prepared and probably produce better quality exams. 

By participating in the study, I hope I have contributed something of 
worth and value to prospective students in the NAU/CEE Educational 
Leadership doctoral program. 
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Recommendations 



Many recommendations for improvements were generated by this 
research project* It is the group's hope that these recommendation will be read 
and utilized to improve the NAU/CEE doctoral program in Educational 
Leadership* 

1* Hold interviews at off-campus locations. 

2. Provide a uniform packet of program requirements to all students and 
faculty. 

3. Make the admissions process consistent. 

4* Provide feedback to students at all points of the admission process* 
5* Clarify the admission process of CEE and the Graduate College. 

6. Provide adequate advisement and /or seminars on the admission 
process. 

7. Increase the number of courses offered in qualitative research. 

8. Increase the use of computer statistic programs in statistic courses. 

9. Decrease the secretiveness about comprehensive exams by creating a 
step-by-step handbook on content and requirements. 

10. Eliminate in-house/timed portions of comprehensive exams. Replace 

with take-home comprehensive exercises. 

11. Create an up-to-date data-base on students' progress toward degree. 

12. Provide additional off-campus assistance for dissertations. 

13. Increase the number of courses offered at off-campus locations. 

14. Add a pre-dissertation seminar at the beginning of the program. 

15. Clarify sequence of courses needed for successful completion of the 

program. 

16. Clarify how the residency requirements can be fulfilled. 
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Implications 
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Educational researchers have historically used personal interviews and 
questionnaires to collect research data. The focus group approach allows for in- 
depth discussion and probing to obtain those all-important divergent points of 
view. In this manner, meaningful insight into group members' perceptions about 
the topic can be gleaned (Kiieger, 1988). This project confirmed the usefulness of 
focuj group research methodology. 

In regards to doctoral students' experiences in the CEE program in 
Educational Leadership, there is an imbricated barrier of communication 
between the students, faculty and administrators. This encompassing 
communication breakdovm is found throughout all areas in this study; 
Uniformity of course requirements, the admissions process, advisement policy, 
information regarding the comprehensive examinations, course sequencing, and 
residency requirements. 

Since communication skills are one of the 12 areas that doctoral students are 
encouraged to become competent, it would seem wise to encourage better 
communication within the Educational Leadership program. 

The Educational Leadership program's lack of communication to its 
doctoral students carries over into its approach to the education provided. In 
spite of the collaborative learning model espoused by the Center for Excellence in 
Education, many of the classes are still taught vis-a-vis traditional teacher 
centered models. AddiHonally, many classes are mislabelled as seminars when 
they have enrollments of more than fifty students. A lack of applied pedagogy is 
demonstrated in many classes as the shidents are treated like children rather 
than adults. 

It is hoped that the results of this study will be used in a positive manner 
to improve the quality of the Educational Leadership program at NAU/CEE. 

51 



LITERATURE CITED 



Glesne, C. & Peshkin, A. (1992). Becoming qualitative researchers: An 
introduction . New York: Longman. 

Krueger, R.A. (1988). Focus groups: A practical guide for applied resear ch. 
Newbury Park, CA: Sage. 

Miles, M.B. & Huberman, A.M. (1984). Qualitative data analysis: A sourcebook 
of new methods . Beverly Hills, CA: Sage. 

Morgan, D.L. (1988). Focus groups as qualitative research . Newbury Park, CA: 
Sage. 

Packard, R.D. & Dereshiwsky, M.L (August, 1990). An evaluation of key 

components of an education doctoral program: Results of a focus group 
teaching si m ulation for education doctoral students . Paper presented to 
the Center for Excellence in Education doctoral program. Flagstaff 
Arizona: Northern Arizona University. 

Patton, M.Q. (1990). Qualitative evaluation and research methods . 
Newbury Park, CA: Sage. 

Schofield, J.W. (1989). Increasing the generalizability of qualitative research. In 
E.W. Eisner & A. Peshkin (eds.5. Qualitative inquiry in education: The 
continuing debate (pp. 200 - 225). New York: Teachers College Press. 

Strauss, A. & Corbin, J. (1990) Basics of qualitative research: Grounded theory 
procedures and techniques . Newbury Park, CA: Sage. 

Yin, R.K. (1989). Case study research: De si gn and methods . Newbury Park, CA: 
Sage. 



42 



